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PREFACE 


The Agriculture and Resources Inventory Surveys Through Aerospace Remote 
Sensing is a multiyear program of research, development, evaluation, and 
application of aerospace remote sensing for agricultural resources, which 
began in fiscal year 1980. This program is a cooperative effort of the 
U.S. Department of Agriculture, the National Aeronautics and Space 
Administration, the National Oceanic a-’,d Atmospheric Administration 
(U.S. Department of Commerce), the Agency for International Development 
(U.S. Department of State), and the U.S. Department of the Interior. 

The work which is the subject of this document was performed by the Earth 
Resources Applications Division, Space and Life Sciences Directorate, 

Lyndon B. Johnson Space Center, National Aeronautics and Space Administration 
and Lockheed Engineering and Mangement Services Company, Inc., were accompli- 
shed under Contract NAS 9-15800. 
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1. INTRODUCTION 


Because of a March 1981 reduction in the National Aeronautics and Space 
Administration (NASA) budget for the Foreign Commodity Production Forecasting 
(FCPF)^ project of the Agriculture and Resources Inventory Surveys Through 
Aerospace Remote Sensing (AgRISTARS) program, the fiscal year (FY) 1982 
UaS.S.R. Barley Pilot Experiment was cancelled; also as a result, the 1981 
exploratory experiment has been replanned for completion in FY 1984, Prior to 
rescoping FCPF project tasks, a significant amount of developmental system 
implementation activity was in the final stages of preparation. The U.S.S.R. 
indicator regions (IR's) and the U.S. foreign similarity regions (FSR's), 
which support the experiment design function, had been defined and docu- 
mented. Data and data systems requirements had been formulated, and 
experiment provisioning was in progress. The preliminary experiment design 
was near completion. Crop labeling procedures development for the U.S.S.R. 
barley regions was finished, with only documentation remaining to be 
completed. Other technology development associated with this foreign explo- 
ratory experiment - specifically, the automated barley separation (labeling) 
technique and- vhe error model - was progressing as scheduled. 

This summary outlines the highlights of the work which was accomplished under 
each experiment subcomponent. Other significant unpublished exploratory 
experiment materials have been incorporated into the appendixes. The experi- 
ment schedules as of April 1, 1981, are shown in figure 1-1. 


Us of January 19, 1982, the project name and objectives were changed. The 
new name is Inventory Technology Department (ITD). 
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Figure 1-1.- FCPF project schedules for 1981, U.S.S.R. Barley Exploratory Experiment. 











FOREIGN COMMODITY PRODUCTION FORECASTING (FCPF) PROJECT SCHEDULES 
1981 USSR BARLEY EXPLORATORY EXPERIMENT (CONTINUED) 
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2. EXPERIMENT DESIGN 


2.1 DEFINED IR's (figure 2-1, ref, 1) 

• High barley regions 

- Belorussia 

- Central Region (added after original IR selection) 

• Barley/spring wheat region 

- Ural Region 

• Barley /winter wheat region 

- North Caucasus Region 

2.2 DEFINED FSR's (figure 2-2, ref. 2) 

• High barley region 

- Agrophysical unit (APU) 104 Montana 

• Barley/spring wheat region 

- APU 20 North Dakota and Minnesota 

- Southern Manitoba and Saskatchewan 

• Barley /winter wheat regions 

- APU 23 Montana 

- Whitman County, Washington 

- Bannock, Franklin, Oneida, and Pierce Counties, Idaho 

2.3 PRELIMINARY DESIGN (See appendix A for detailed information.) 

• Assessment of available data completed 

• Assessment of current technology’ completed 

• Preliminary plans in work 

f FCPF Project Management briefing materials in work 
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HIGH BARLEY 2 - BARLEY/SPRING WHEAT 

A. Belorussia Region Ural Region 

B. Central Region 

3 - barley/hinter wheat 

North Caucasus Region 





3. DATA REQUIREMENTS 


• Level A experiment requirements completed (figure 3-1) 

• Initial Level B experiment requirements completed. (Detailed require- 
ments are in appendix B.) 

• Final Level B experiment requirements are in work. 

- Dropped yield requirements 

- Chose a subset of the originally requested Landsat data (figures 3-2 
through 3-6). 
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STANDARD DATA ScTS 



Figure 3-1.- Standard data sets. 
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Figure 3-3.- Barley/spring wheat IR, Ural Region. 




3-4 



Figure 3-4.- Barley/winter wheat IR, North Caucasus Regi 






Figure 3-6.- High barley IR, Central Region. 



4. SAMPLING 


Reallocation of 1981 Landsat segments for the U.S.S.R. FSR's (see appendix C) 
was completed; the reallocation was done to emphasize the study of barley 
(ref. 3). 
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5. EXPERIMENT PREPARATION 


• U.S.S.R. meteorological data were procured, 

- Meteorological data from 1969 through I960 

- First order station collection 

- Contains daily maximum and minimum temperature and amounts of precipi- 
tation 

- Contains adequate information to create required crop calendar and 
weather analysis products 

• Meteorological data bases were set up for 1976 and 1977 for the IR's. 

• Biowindow midpoint model was run for the 1977 IR segments. 

• Generation of the other U.S.S.R. crop calendar and weather analysis pro- 
ducts was postponed after the announcement of the budget reduction. 

• Crop calendar and weather analysis products for the FSR segments were 
generated because these segments were also required for the U.S. /Canada 
Spring Small Grains (SSG) Pilot Experiment. 


5-1 


6. SEGMENT ANALYSIS 


• Labeling development task, consisting of the following study areas, 

completed : 

- Verification of the baseline SSG procedure for use in the U.S.S.R. and 
recommendation of necessary changes 

- Modification of existing strategy for separating winter small grains 
from spring small grains 

- Recommendations -on the use of currently available barley separation 
techniques in the U.S.S.R. 

- Studies and recommendations on the applicability of current crop cal- 
endar and biowindow midpoint models to U.S.S.R. data 

- Results and recommendations for documentation of the labeling develop- 
ment task in progress (ref. 4) 



7. AGGREGATION 


The U.S.S.R, historical at-harvest data base was procured and digitized. 


8. ACCURACY ASSESSMENT 


t Initial development of the error model is completed. 

t Testing of the model is planned to occur in the U.S. SSG Pilot 
Experiment. 
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9. ASSOCIATED TECHNOLOGY DEVELOPMENT 


• Development of a barley separation procedure is in progress within the ITD 
project technology development task. 

- The procedure uses the pixels (picture elements) labeled spring small 
grains in combination with meteorological variables to separate the 
barley from other spring small grains, 

- Software for this procedure has been completed. 

- Definition of coefficients remains to be completed. 

- Plans are to continue development and testing. 
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10. CONCLUDING REMARKS 


Barley separation results in the U.S.S.R. Exploratory Barley Experiments 

are anticipated to be similar or worse than those in U.S. Transition Year 
(TY) experiments. 

- Sampling in both areas optimizes wheat (not barley) identification. 

- Only 18-day acquisition histories available in the IR's. 

- Different crop mix exists. 

- Drought stress was detected in U.S.S.R. SSG/barley IR. 
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PRELIMINARY EXPERIMENT DESIGN 


APPENDIX A 

ASSESSMENT OF FY 1981 U.S.S.R. BARLEY EXPLORATORY EXPERIMENT 

A.l PRELIMINARY EXPERIMENT DESIGN 
A. 2 TECHNOLOGY ASSESSMENT 


A. 3 DATA ASSESSMENT 


PRELIMINARY EXPERIMENT DESIGN 


APPENDIX A 

ASSESSMENT OF FY 1981 BARLEY EXPLORATORY EXPERIMENT 

Appendix A consists of three major divisions. The appropriate division 
heading is printed at the top of each page in this appendix. All data are 
categorized within one of these major divisions with subheadings numbered 
according to standard documentation format. 

A.l PRELIMINARY EXPERIMENT DESIGN 
A. 1.1 GENERAL OBJECTIVES 

• Evaluation of an at-harvest SSG/barley proportion estimation 
technology developed in the United States and adapted to a foreign 
country. 

• Definition of a U.S.S.R. barley baseline procedure by evaluating and 
comparing results of alternative technologies. 

• Evaluation of the first method of assessing proportion estimation 
accuracy in the event that ground observations are unavailable. 

• Evaluation in the U.S. of the effects of known U.S.S.R. data 
restrictions on multiyear aggregation procedures. 

A.l. 2 SPECIFIC OBJECTIVES 

• To determine the performance (in terms of labeling accuracy and 
proportion estimation accuracy) of the adapted SSG2 procedure. 

• To determine the effect of automating the barley labeling component 
of the adapted SSG2 procedure. 

• To determine the effect of having a manual pasture filter component 
as contrasted with having none in the adapted SSG2 procedure. 

• To determine the effect of automating the pure-pixel identification 
module of the adapted SSG2 procedure. 

• To determine the effect of automating the acquisition selection of 
the adapted SSG2 procedure. 
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PRELIMINARY EXPERIMENT DESIGN 


• To determine the performance (in terms of proportion estimation 
accuracy) of the following procedure. 

- The SSG4 (spatial /col or sequence) procedure with complete 
enumeration of spring small grains. 

- The SSG4 (spatial /col or sequence) procedure with the automated 
barley labeler and the three-category proportional Bayesian 
estimation technique. 

• To evaluate state-of-the-art accuracy assessment methodology for a 
foreign area where little or no ground observation Information Is 
available. 

I- 

it 

• To evaluate state-of-the-art adaptations of multiyear aggregation 
procedures Incorporating a multiyear model and an updated grouping 
logic to a foreign area. 

- The effect of restricted multiyear data on aggregations. 

- The effect of limited historical data on aggregations. 

• To evaluate the effect of the biowindow midpoint model. 

A. 1.3 SCOPE 

• Areas^ of study' (tables A-1 and A-2; figures A-1 through A-7) 

- 25 FSR segments 

- 20 IR segments 

• Crop years 

1976-80 for mulltyear aggregation 
1976-79 for FSR evaluations 
1977 for IR evaluations 


Vlt Is necessary to have both FSR and IR segments to evaluate the spring 
small grains error model. 
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Segment 

number 


1102 
1104 
1389 
1553 
“1725 



1945 

1473 

“1513 

1514 

151J 

“1521 

1523 


3130 

3144 

“3169 

3179 

“3197 


High barley 
High barley 
High barley 
High barley 
High barley 

High barley 
High barley 
Barley/winter wheat 
Barley/winter wheat 
Barley/wi nter wheat 

Barley/winter wheat 
Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 

Barley/spring wheat 
Barley/spnng wheat 
Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 

Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 


A1 ternates: 


3050 

“3083 

3132 

3166 


High barley 
Barley/winter wheat 
Barley/spring wheat 
Barley/spring wheat 

Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 
Barley/spring wheat 


State or 
Province 



“indicates segment with more than one year of data 
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TABLE A-2.- IR DATA SET - U.S.S.R. BARLEY EXPLORATORY EXPERIMENT 



Segment 

number 

1 

FSR 

Oblast 

*7045 

High barley 

Gomel 

7500 

High barley 

Bryansk 

7513 

High barley 

Vladmir 

7543 

High barley 

Moscow 

8040 

High barley 

Orel 

7343 

Barley/winter wheat 

Stavropol 

7377 

Barley /winter wheat 

Rostov 

7379 

Barley/winter wheat 

Rostov 

7382 

Barley/winter wheat 

Rostov 

7395 

Barley/winter wheat 

Rostov 

7853 

Barley/winter wheat 

Stavropol 

7880 

Barley/winter wheat 

Rostov 

7955 

Barley/spring wheat 

Orenburg 

7966 

Barley/spring wheat 

Orenburg 

7967 

Barley/spring wheat 

Orenburg 

7971 

Barley/spring wheat 

Orenburg 

7995 

Barley/spring wheat 

Orenburg 

8762 

Barley/spring wheat 

Orenburg 

8773 

Barley/spring wheat 

Orenburg 

ternat 

es: 


7533 

High barley 

Kaluga 

7544 

High barley 

Tula 

7565 

High barley 

Smolensk 

8050 

High barley 

Orel 

7962 

Barley/spring wheat 

Orenburg 

7990 

Barley/spring wheat 

Orenburg 

8816 

Barley/spring wheat 

Orenburg 

8830 

Barley/spring wheat 

Orenburg 

9437 

Barley/spring wheat 

Orenburg 

7361 

Barley/winter wheat 

Stavropol 

7347 

Barley/winter wheat 

Rostov 

7389 

Barley/winter wheat 

Rostov 

7792 

Barley/winter wheat 

Stavropol 

7849 

Carley/winter wheat 

Stavropol 

7882 

Barley/winter wheat 

Rostov 



^Indicates segment with more than one year of data. 
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ORIGINAL PAGE ^ 

POOR QUALITY U.S.S.R. SIMILARITY REGIONS 



2 • barley/spring wheat 

Primary: APJ 20 North Oakota/MInnesota 

Secondary: Southern Manitoba/SaskatcheMan, 

Canada 

3 • BARLEY/WINTER WHEAT 

Primary: APU 23 Montana 

Secondary: Whitman County, Washington 

Tertiary: Bannock, Franklin, and Oneida 

Cour.tles, Idaho 

Figure A-l.- FSR's for the U.S.S.R. 
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Figure A-2.- FSR data set, U.S.S.R. Barley Exploratory Experiment 
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Figure A--3.- Defined IR' 
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Figure A-4.- High barley IR, Central Region 
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Figure A-6.- Barley/winter wheat IR, North Caucasus Region 
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A. 1.4 APPROACH 

• Pi'oportion estimates will be generated for segments in the FSR's for 
the adapted SS62 procedure and four modifications to it. 

- The modifications are either the automation or exclusion of a 
manual subcomponent 

M First, an automated barley labeling subcomponent will be 
substituted for the manual barley procedure. 

•• Second, (in addition to the automated barley labeler) no 
pasture filtering task will be performed by analysts; the 
pasture filter is a labeling step performed by analysts to 
reduce commission errors. 

•• Third, (in addition to the automated barley labeler and no 
use of the pasture filtering task) an automated pure-pixel 
selection subcomponent will be substituted for the task 
performed by analysts. 

Pure pixels are used in labeling. When a boundary pixel is 
encountered, an alternate pure dot is associated with it and 
used for labeling. 

•• Fourth, an automated acquisition selection procedure will be 
used (in addition to the automated barley labeler, no use of 
the pasture filtering task, and an automated pure-pixel 
selection subcomponent). 

~ The semiautomated acquisition selection procedure requires 
analysts to check the automatically selected acquisitions 
for appropriateness. 

- These modifications were selected to determine the effects of the 

automated versions of the subcomponents at a minimal expense. 
There are 32 possible combinations. Proportion estimates cannot 
be generated for all configurations. Only five were used. By 
this design, intermediate analyst results from the SSG2 procedure 
can be used for input into automated subcomponents so additional 
analyst input will not be required. " 
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• Proportion estimates will be generated for segments in the IR's for 
the adapted SSG2 procedure. 

• Proportion estimates will be generated for segments in the FSR's and 
IR's fop the spatial /col or sequence (SSG4) procedure. 

- SSG dots used in the baseline will be given SSG4 labels and then 
be put through the automated barley labeler. 

• Proportion estimates will be generated using ground-truth (GT) 
labels (barley, other spring small grains, or nonsmall spring 
grains). 

• , Error model will be applied to. segments. 

• North Dakota segments will be aggregated for area estimation based 
on GT labels and limited historical data and restricted multiyear 
data. 

A. 1.5 METHODOLOGY 

• Process 25 FSR and 20 IR segments with the adapted SSG2 procedure. 

- Configuration 1 (fig. A-8) 

Process segments using: semiautomatic acquisition selection, 

manual pure-pixel selection, analyst pasture filter (manual 
subcomponent), adapted reformatted SSG labeling, manual barley 
labeling, and three-category proportional Bayesian estimation 
technique. 

• Process 25 FSR segments with baseline procedure modified by 
automated components. 

- Configuration 2 (fig. A-8) 

Process segments using: semiautomatic acquisition selection, 

manual pure-pixel selection, analyst pasture filter (manual 
subcomponent), adapted reformatted SSG labeling, automated barley 
labeling, and three-category proportional Bayesian estimation 
technique. This is done to determine the effect of the automated 
barley labeler; it requires no additional analyst contact time. 
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Configuration 3 (fig. A-8) 

Process segments using: semiautomatic acquisition selection, 

manual pure-pixel selection, no pasture filter, adapted 
reformatted SSG labeling, automated barley labeling, and three- 
category proportional Bayesian estimation technique. This is 
done to determine the additional effect of having no pasture 
filter; it requires no additional analyst contact time. 

Assumption: The pasture filter will perform as well with manual 

pure dots as with automated pure dots. 

- Configuration 5 (fig. A-8) 

Process segments using: automated acquisition selection, 

automated pure-pixel identification, no pasture filter, adapted 
reformatted SSG labeling, automated barley labeling, and three- 
category proportional Bayesian estimation technique. This is 
done to determine the additional effect of the automated 
acquisition selection component; it requires no additional 
analyst contact time. 

t Process 25 FSR and 20 IR segments with the SSG4 (spatial /col or 
sequence) procedure. 

- With complete enumeration for SSG 

- With dots put through the barley labeler and three-category 
proportional Bayesian estimation technique. 

• Process 25 FSR segments using GT labels (barley, other spring small 
grains, and nonsmall grains) and the three-category proportional 
Bayesian estimation technique. 

• Process 25 FSR and 20 I R segments using the integrated procedure. 

• Apply the error model to the FSR segments and to the IR segments. 
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In North Dakota, aggregate area estimates of 1976-80 

- Produce aggregations using GT labels and varying quantities of 
historical data. 

- Produce aggregations using GT labels and varying quantities of 
available past years' data. 

Use 1976, 77, 78 data in multiyear grouped optimal aggregation 
technique (MYGOAT) to predict 79 estimates. 

Use 1976, 77, 79 data in MYGOAT to predict 78 estimates. 

Use 1976, 78, 79 data in MYGOAT to predict 77 estimates. 

Use 1977, 78, 79 data in MYGOAT to predict 76 estimates. 


Use 

1976, 77 

data 

in 

MYGOAT 

to 

predict 

79 

estimates. 

Use 

1976, 77 

data 

in 

MYGOAT 

to 

predict 

78 

estimates. 

Use 

1976, 78 

data 

in 

MYGOAT 

to 

predict 

79 

estimates. 

Use 

1976, 78 

data 

in 

MYGOAT 

to 

predict 

77 

estimates. 

Use 

1976, 79 

data 

in 

MYGOAT 

to 

predict 

78 

estimates. 

Use 

1976, 79 

data 

in 

MYGOAT 

to 

predict 

77 

estimates. 

Use 

1977, 78 

data 

in 

MYGOAT 

to 

predict 

79 

estimates. 

Use 

1977, 78 

data 

in 

MYGOAT 

to 

predict 

76 

estimates. 

Use 

1977, 79 

data 

in 

MYGOAT 

to 

predict 

78 

estimates. 

Use 

1977, 79 

data 

in 

MYGOAT 

to 

predict 

76 

estimates. 

Use 

1978, 79 

data 

in 

MYGOAT 

to 

predict 

76 

estimates. 

Use 

1978, 79 

data 

in 

MYGOAT 

to 

predict 

77 

estimates. 


Use 

1976 

data 

in 

MYGOAT 

to 

predict 

77 

estimates. 

Use 

1976 

data 

in 

MYGOAT 

to 

predict 

78 

estimates. 

Use 

1976 

data 

in 

MYGOAT 

to 

predict 

79 

estimates. 

Use 

1977 

data 

in 

MYGOAT 

to 

predict 

76 

estimates. 

Use 

1977 

data 

in 

MYGOAT 

to 

predict 

78 

estimates. 

Use 

1977 

data 

in 

MYGOAT 

to 

predict 

79 

estimates. 

Use 

1978 

data 

in 

MYGOAT 

to 

predict 

76 

estimates. 

Use 

1978 

data 

in 

MYGOAT 

to 

predict 

77 

estimates. 

Use 

1978 

data 

in 

MYGOAT 

to 

predict 

79 

estimates. 

Use 

1979 

data 

in MYGOAT 

to 

predict 

76 

estimates. 
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Use 1979 data in MYGOAT to predict 77 estimates. 

Use 1979 data in MYGOAT to predict 78 estimates. 

t There are 28 combinations (seven combinations of years to predict 
each of the four years' aggregations). 

• All statistics for the 28 combinations can be computed with only 10 
runs of the MYGOAT. 

• Aggregate for wheat, barley, and oats for combinations not 
containing 1976 data; otherwise, aggregate for wheat only. 

A. 1.6 EVALUATIONS 
A. 1.6.1 FSR Segments 

• For the baseline, determine the bias, standard deviation, and mean 
square error (MSE) of the proportion estimates and evaluate 
labeling accuracy. 

• For the automated barley labeler, determine the effect on 
proportion estimation accuracy and labeling accuracy by a 
comparison to those of the baseline (configuration 2 compared to 
configuration 1). 

• For the pasture filter, determine the additional effect of having 
no pasture filter on proportion estimation accuracy and labeling 
accuracy by a comparison to those when adding only the automated 
barley labeler (configuration 3 compared to configuration 2). 

• For the automated pure-pixel identification module, determine the 
additional effect on proportion estimation accuracy and labeling 
accuracy by a comparison to those when adding only the automated 
barley labeler and no pasture filter (configuration 4 compared to 
configuration 3). 

• For the automated acquisition selection, determine the additional 
effect on proportion estimation accuracy and labeling accuracy by 
a comparison to those when adding the automated barley labeler, no 
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pasture filter, and the automated pure-pixel identification module 
(configuration 5 compared to configuration 4). 

• For the-three category proportional Bayesian estimation technique, 
determine (a) the accuracy and precision when GT labels are input; 
(b) the effect in relation to random sa;npling and; (c) the effect 
of the Bayesian estimator by a comparison to the estimates of the 
proportional maximum likelihood estimator (MLE). Then construct a 
generalized beta prior, based on empirical results, and compare 
with the prior used. 

• For the SSG4 (spatial/color sequence SSG) procedure, determine the 
accuracy and precision of proportion estimates. 

For the dots used in the baseline but processed with field 
destgnations/labels of the SSG-4 procedure, the automated barley 
labeler and the three-category proportional Bayesian estimation 
technique, determine the accuracy/ precision of proportion 
estimates and the labeling accuracy. Compare these results with 
the SSG2 procedure results. 

• For the error model, determine the accuracy/precision in estimat- 
ing errors of the SSG2 procedure. Compare the results with those 
that were obtained in the development, using the SSG2 procedure 
for the United States to determine the effect of the adaptation 
(procedure not adapted to the U.S.S.R.). 

• For the aggregations, determine the accuracy of areal estimates as 
compared to U.S. Department of Agriculture (USDA) estimates. 
Determine the effect that varying quantities of historical or 
multiyear data have on the decision of aggregated estimates. 

.1.6.2 IR Segments 

• For the adapted SSG2 procedure, determine the effect on proportion 
estimates because of no pasture filter. 

• For the biowindow midpoint model, determine a quantitatively 
defined biowrndow in terms of greenness and brightness, and 
compare with spectral biowindrw model predictions. 
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If the acquisition date falls outside the spectral biowindow, 
determine the deviation in days of the acquisition from the 
nearest endpo it of the window. Negative values indicate that the 
biowindow model predictions were early; positive values indicate 
they were late. 

A. 1.7 QUESTIONS CONCERNING THE PROCEDURES 

A. 1.7.1 Reformatted Procedure 

• What is the labeling accuracy of cropland/noncropland? 

• What is the labeling accuracy of grain/nongrain? 

t What percentage of grain is omitted to noncropland and what 
percentage of grain is omitted to nongrains? 

• What are the sources of labeling errors? 

-Noncropland (water, pasture, etc.) 

-Nongrain (corn, soybeans, flax, etc.) 

• Do labeling errors and their sources vary from one crop-mix region 
to another? 

• Can certain logic paths in the labeling decision be associated 
with certain types of errors? 

• Will a comparison of machine labels with the integrated analysis 
labels for IR segments show qualitative similarities with FSR 
segment results? 

A. 1.7. 2 Reformatted Barley Procedure and the Automated Barley Procedure 

§ What is the labeling accuracy of barley? 

• What is the relationship (with regard to labeling accuracy of 
barley) between the amount of barley relative to the amount of 
other small grains? 


A- 19 


PRELIMINARY EXPERIMENT DESIGN 


• Would we do better to plot all SSG dots [i.e., (the number of SSG 
dots) X (number of segments)] on the same green number versus 
brightness plot for the purpose of separation? 

• Would a linear discriminant function be superior to the 
fixed-slope approach to separation (reformatted only)? 

• What percentage of barley is omitted as noncropland? 

• What percentage of barley is omitted as nongrain? 

• What percentage of barley is unseparable? 

• What nongrain pixels are committed with barley? 

• What is the nature of Q and V labels? 

«• How "definite" is definite barley and definite other small grains? 

• Does labeling accuracy differ accross the crop-mix areas? 

f Is increased accuracy achieved with 418 dots? 

A. 1.7. 3 Automated Boundary Pixel Selector 

• How many boundary pixels are missed? 

• How many boundary pixels are in interiors? 

• Does the selector perform as well as the manual procedure and if 
not, is the selector still justified due to increased efficiency? 

• What is the effect of the reporting level of historical data upon 
area estimation? 

• What is the effect of limited historical data upon area estimation? 

• What is the effect of limited multiyear data upon area estimation? 

• What is the effect of cloud cover upon area estimation? 

• What is the effect of the lack of precision of within- stratum 

variance estimates upon area estimation? 
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A. 1.7. 4 Spatial /Color Sequence Proportion Estimation 

• What is the purity of the extracted fields? 

• How closely do the extracted fields correspond to ground- truth 
fields? 

• Which biowindow sequences tend to be the most accurate predictors 
of- SSG, and what are the characteristics of those fields whose 
biowindow sequence is not a reliable predictor of SSG? 

• How effective is the 2-4-4 biowindow sequence In the 
identification of barley? 

• Of the small grains omitted, what percentage is barley? 

• What biowindow sequences tend to lead tO' omission, and which tend 
to lead to commission? 

• How does proportion estimation compare with Bayes proportion 
estimation? 

A. 1.7. 5 Bayes Estimation, Probability of Correct Classification/Exhaustive 
Search, and Probability of Correct Classification Adjustment for 
La bel Errors 

• General 

- What is the mean bias and its variance over all 25 FSR segments 
and within each crop-mix area? 

- What portion of the bias is due to labeling, to procedure, and 
to sampl i ng? 

- What portion of the MSE is due to bias? 

- How do machine-generated proportion estimates compare to 
relative count? 

- How do these procedures compare? 

• Probability of correct classification adjustment for label errors 

- How accurate are the labeling error estimates? 
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A. 1.7. 6 Error Model 

• How does the predicted stratum level mean bias compare with the 
calculated statum level mean bias? 

• How does the predicted segment level error compare with the actual 
segment level error? 

• Which level of estimation - segment or stratum - yields the most 
reliable and most useful estimate of proportion estimation error? 

A, 1.7. 7 Biowindow Midpoint Selection/Acquisition Selection Design 

• Does the biowindow midpoint/acquisition selector assign acquisi- 
tions to the same biowindow as the subjective procedure? 

A. 1.7. 8 Overall Performance 

• Given the performance of technology in the U.S.S.R. Barley 
Exploratory Experiment, what performance could be expected in the 
U.S.S.R. at the country level (yield, area, production)? 

- In accuracy 

- In precision 

•• For winter grains 
M For spring grains 
•• For all small grains 
M For barley 
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A.2 TECHNOLOGY ASSESSMENT 

The SSG2 labeling procedure is presented in figure A-9. 

A.2.1 ADAPTED SSG2 PROCEDURE 

The SSG2 procedure performance results prior to the new adaptation were as 

follows: 

f 30 segments, U.S. Northern Great Plains (USNGP), 1976-1979 (1981 U.S. /Canada 

SSG Pilot Experiment) 

- Mean error = 2.0 

- Standard deviation = 7.35 

Adapting the SSG procedure for application in the U.S.S.R. required solving 

some problems. This effort was proposed to compensate for the U.S.S.R. 

labeling difficulty. The problems and solutions are presented herein. 

Proposed actions to compensate for U.S.S.R. labeling problems: 

Problem #1 : Commission of large numbers of winter grain dots to spring grain 

in the Belorussia, North Caucasus, and Central Regions. 

Sol uti on : The Belorussia and North Caucasus Regions require a Window 1 

acquisition as part of the minimum data set for both the basic and 
augmented logic. In the Central Region where spring and winter 
grains are not separable, slight adjustment to the window midpoints 
is done to ensure the labeling of both winter and spring small 
grains; the SSG logic is used. Mathematical methods will be 
required to directly proportion the winter grains, spring grains, 
and barley in this region to determine the relative acreages. 

Problem #2 ; Omission of large numbers of SSG dots in the Ural Region because 
of the 1977 drought (lack of "green-up"). 
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Figure A-9.- The SSG2 labeling procedure flow chart. 
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Solution ; For the 1977 drought segments, in the Ural Region only, (a) lower 
the window 2 green number threshold from >8 to >5 in both the basic 
and augmented logic and (b) apply the augmented labeling logic 
first, omitting module C; process with basic logic only if not 
workable using this modified augmented logic. 

Problem #3 ; Lack of barley separation in a single acquisition scatter plot 
(as previously observed in the United States). No knowledge of 
the effect of drought (Problem 2) on barley separation is also 
related. 

Sol uti on ; Assume that all SSG in the North Caucasus Region is barley and 

aggregate accordingly. Separate barley in the Ural and Belorussia 
Regions using the dot drift version of the U.S. barley separation 
procedure where possible and treating all SSG dots as "Q" dots. 
Separate barley in the other segments by placing a line in the 
single acquisition plot by interpreting colors, using segments 
separated by the dot drift method in the same area for color 
reference. 

Listed below are the detailed changes to the baseline labeling procedure: 

Belorussia Region 

• Processing logic order of precedence: 

- Basic logic 

- Augmented logic 

• Minimum data sets for basic logic: 

- Windows 1, 2, 3, 4 

- Windows 1, 2, 4 

. - Windows 1,3,4 
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• Changes in modules/data sets for augmented logic: 

- Shorten the length of time period 5 from 15 days to 8 days. 

- Delete option to process with time period 0; window 1 is required. 
Minimum data set becomes at least one acceptable acquisition from each 
of the following groups: 

•• Group 1: window 1 

#• Group 2: window 2, period 2-3, or window 3 

M Group 3: period 3A or period A 

Central Region 

• Processing logic order of precedence: 

- basic logic 

- augmented logic 

• Minimum data sets for basic logic: 

- windows 1, 2, 3, 4 

- windows 1, 2, 4 

- windows 1, 3, 4 

• Changes in modules/data sets for augmented logic: 

- Shorten length of time period 5 from 15 days to 8 days 

- Minimum data set for augmented logic is at least one acceptable 
acquisition from each of the following groups: 

•t Group 1: period 0, window 1, window 4 or period 5 

•• Group 2: window 2, period 2-3 or window 3 

•• Group 3: period 3A or period A 

• Manual midpoint verification of each segment to assure accomodation of the 
variability of all small grains, both winter and spring. (This is a 
verification unique for this region.) 
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• Barley separation procedure does not apply. Statistical methods will be 
required to arrive at a barley proportion. 

North Caucasus Region 

• Processing logic order of precedence: 

- Basic logic 

- Augmented logic 

• Minimum data sets for basic logic: 

- Windows 1, 2, 3, 4 

- Windows 1, 2, 4 

- Windows 1, 3, 4 

• Changes in modules/data sets for augmented logic: 

- Shorten length of time period 5 from 15 days to 8 days. 

- Delete option to process with time period 0; window 1 is required. 
Minimum data set becomes at least one acceptable acquisition from each 
of the following groups: 

M Group I: window 1 

M Group 2: window 2, period 2-3, or window 3 

•• Group 3: period 3-A or period A 

f Barley separation procedure is not applicable in this region. All spring 
small grains should be labeled barley. 

Ural Region 

• Processing logic order of precedence (1977 crop year only): 

- Augmented logic 

- Basic logic 

• For other crop years: 

- Basic logic 

- Augmented logic 
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0 Minimum data sets for basic logic: 

- Windows 1, 2, 3, 4 

- Windows 1, 2, 4 

- Windows 1, 3, 4 

0 Changes in modules/data sets for augmented logic: 

- Shorten length of time period 5 from 15 days to 8 days. 

- Process with either time period 0 or window 1; minimum data sets are as 

defined for the U.S. baseline procedure. 

- For 1977 segments in Chelyabinsk, Kurgan, and Orenburg Oblasts, bypass 
module C logic. This will partially compensate for drought effects in 
these areas. 

- For the 1977 crop year in Chelyabinsk, Kurgan, and Orenburg Oblasts, 
lower the green number threshold for window 2 in module G from > 8 
to > 5. This will partially compensate for drought effects in these 
areas. 

0 Barley separation procedure is applicable in this region. 

Table A-3 presents the data derived when the new SSG2 procedure was applied to 

U.S.S.R. results. 


TABLE A-3.- SSG2 PROCEDURE ADAPTATION TO U.S.S.R. RESULTS 


Segment 

number 

Logic 

Region/ 

oblast 

Procedure 

SSG2 

Adapted 

Procedure 

SSG2 

7950 

Augmented 

Orenberg 

0.00 

2.12 

7959 

Augmented 

Orenberg 

-4.54 

3.40 

7973 

Augmented 

Orenberg 

-8.23 

-1,17 

8021 

Augmented 

Chelyabinsk 

-62.16 

-8.10 

7950 

Basic 

Orenberg 

10.63 

12.76 

7959 

Basic 

Orenberg 

0.00 

15.90 
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A.2.2 THREE-CATEGORY PROPORTIONAL ALLOCATION/BAYESIAN ESTIMATOR PROPORTION 
ESTIMATION TECHNIQUE 

A.2.2. 1 General 

• Barley proportion estimation by historical ratioing small grains estimates 
was found too inaccurate and requires accurate historical statistics, 

• Previous techniques used machine proportion estimation for spring small 
grains-nonsmall spring grains and used the manual proportion estimation 
technique for barley-other small grains. 

• The objective is to present a machine proportion estimation technique for 
simultaneously estimating, in one step, barley, other small grains, and 
nonsmall grains from direct barley-other small grains labels, requiring no 
additional analyst effort. 

The following technique was generated: 

• Scene stratified into clusters 

• Sample dots allocated to clusters proportional to cluster size; 
dots randomly selected within a cluster 

• Bayesian estimates of cluster level proportions 

Dirichlet prior (figures A-10 and A-11) were modeled on 19 segments. 

F(0g,Os) = Ky 0g Ojd - Og " 65^^ 

0g e[0,l] 

ORIGINAL PAGE IS 
0j e[0,l - 0g] OF POOR QUALITY 

-1 < a,b,c, 

; . B + a + 1 

®B “ ir+~a' +1)"+ c T 'J 

" S + b + 1 . 

®S ^ N + a + b + c + 3 

• Aggregated to segment-level estimate by weighting cluster-level estimates 
by cluster size. 
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A. 2. 2. 2 Three -Category Proportional Allocation/Bayeslafi Estimator (PA/BE) 
Test Results 


• Two-category PA/BE technology 

- Beta prior (figures A-12 and A-13) for cluster proportions 

ff«s) = "i - “s)” 

Og e [0, 1] 

-1 < a,b 

®S ■ TTTTia~r2 

• Two-category PA/BE performance 

(1980 U.S. /Canada Wheat and Barley Exploratory Experiment, 35 segments) 




Mean Error (%) 

Sq {%) 

AI labels 

PA/ BE 

-3.5 

6.0 

( Integrated 
procedure) 

Random 

sampling 

-5.7 

7.7 


PA/BE 

0.5 

3.8 

GT labels 

Random 

sampling 

-2.5 

6.9 
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A.2.3 SPECTROMET BARLEY/OTHER SPRING SMALL GRAINS DISCRIMINANT (AUTOMATED 
BARLEY LABELER) TECHNOLOGY (figure A-14) 

• Background 

- Sp<^ctral 3 i^oaration of barley from other spring small grains has been 
observed near the soft dough stage of spring wheat. 

- Segment to segment variation in the position in feature space of the 
barley/other spring small grain discriminant line was hypothesized to be 
the result of the following. 

•• Acquisition history 

§• Meteorological stress 

•• Planting date distribution 

Figures A-15 and A-16 show the overlap of barley and other spring small 
grains using Landsat data alone and with other meteorological data, 
respectively. 

Figures A-17 through A-19 reflect estimates and percentages of barley in 
comparison studies. Table A-4 presents a summary of errors for barley 
proportion. 

• Multivariate normal Bayes linear discriminant function 

G^ (X) = -1/2 (S - 2-1 (X - ) + Log P(W^ ) 

Discriminant function for i^^ group 
Feature space vector 

• Pixel value of greenness/brightness 

• Accumulated degree day for date of acquisition 
Pooled covariance matrix 
Mean 'ector for the i^^ group 
a priori probability for the i^*^ group 


where 

Gl(X) = 
X 


E ■ 
“1 

P(Wj) = 
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TABLE A-4.- SUMMARY OF ERRORS - BARLEY PROPORTION 

Ground truth, % barley 



0 to 2.5 

2.5 to 10.0 

>10.0 

0 to 20.0 

0.0 X 
1.17 Sx 
5.0 N 

0.7 X 
1.02 S.x 
2.0 N 

0.2 X 
.34 Sx 
2.0 N 

Ground truth. 


-.4 X 


X spring small 20.0 to 40.0 


1.04 Sx 


grains 


5.0 N 





-5.9 X 

>40.0 



0.73 Sx 




3.0 N 


Overall: Mean error = 1.04 

Standard 


deviation 


= 2.499 
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nant development 















DISCRIMINANT AXIS 

Figure A-15.- Overlap of barley and other spring small grains 
using Landsat data alone (normalized). 
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of barley - pure dot. 
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A.2.4 AUTOMATED PURE PIXEL SELECTION 

• Labeling is to be done on pure pixels. When a boundary dot is encountered, 
an alternate pure dot is selected for labeling. 

• Configuration of the automated procedure 

- Spatial /col or sequence field delineation (pure and boundary pixel 
designations) 

- Alternate dot selection 

M Search area of 9 (fig. A-20) 

#• Spectral tie option - selects the dot which is spectrally closest to 
original boundary dot 

•• Adapted SSG2 labels for original boundary dot when a pure alternate 
is not found in the search area 

• Automation proposed to reduce analyst input time 

• Tested on 30 segments, USN6P, 1976-79 

The results of the proportion estimation accuracy using 209 random samples and 
ground- truth labels follow: 

• The automated alternate dot selection provides a representative sample. 

• Table A-5 shows the unbiased proportion estimates when ground- truth labels 
are used for input. 

• The selection of alternate dots is appropriate if 70 percent of the 
alternate dots had the same ground- truth crop code as their respective 
(boundary) grid dot. 

The results of proportion estimation accuracy are in table A-6. Sixty dots 
(proportional allocation to CLASSY clusters/Bayesian estimator) and SSG2 
labels (from automated labeling procedure with manual pasture filter) were 
used. SSG2 labels were used for automated selection of dots. 

The comparison of analyst-selected dots with those selected by the automated 
procedure for SSG2 labeling accuracy are given in table A-7. 
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TABLE A-5.- MEAN ERRORS BASED ON GROUND-TRUTH (6T) LABELS 


Error 

209 grid 
dots 

209 grid/ 
alternate 
dots 

Pure grid 
dots 

Mean error, % 

-0.8 

-0.6 

-7.5 

Standard 




deviation 
of errors, % 

2.6 

2.6 

5.4 


TABLE A-6.- MEAN ERRORS BASED ON ANALYST- 
SELECTED AND AUTO-SELECTED DOTS 


Error 

Analyst- 

interpreted 

Automated 

selection 

^Mean error 

2.0 

3.0 

Standard deviation 
of the errors, % 

7.35 

7.61 

Mean absolute error, % 

5.9 

6.7 

Relative mean error, % 

7.5 

11.3 

Mean GT proportion, % 

26.6 

26.6 

Sample size, segments 

30 

30 


^No significant difference between proportion estimates 
when automated pure pixels are labeled and proportion 
estimates when analyst pure pixels are labeled. 
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TABLE A-7.- SSG2 LABELING ACCURACY COMPARISONS FOR ANALYST-SELECTED DOTS 
* AND THOSE SELECTED BY THE AUTOMATED PROCEDURE 


Dots, automated or 
analyst-interpreted 

Smal 1 
grains 

Nonsmal 1 
grains 

Overal 1 

(Both small and 
nonsmall grains) 


^Automated 

75.4 

85.1 

82.7 

Pure grid dots 

**Analyst- 

interpreted 

71.5 

87.6 

83.5 


Automated 

66.9 

82.1 

78.3 

Alternate dots 

Analyst- 

interpreted 

82 .4 

82.6 

82.6 


Automated 

73.3 

84.4 

81.6 

Total dots 

Analyst- 

interpreted 

73.4 

86.9 

83.4 


^24.5 percent of the grid dots were boundary dots according to 
the spatial/ color field delineation. 


**14.5 percent of the grid dots were boundary dots according to 
the analysis. 
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A. 2. 5 AUTOMATED ACQUISITION SELECTION 

• The automated acquisition selection inputs include: 

- Acquisition dates 

- Cloud cover data 

“ Blowindow midpoints 

• Acquisitions ranked in biowindow/periods 

If more than one acquisition falls within a window/period, rank them as 
follows: 


Window/period 


First choice 


1, 2 , 3, A, 4 Closest to the middle of the window/ 

period. (If two acquisitions are 
equidistant from the middle, the latest 
one is the first choice.) 

0 Closest to window 1 


1-2, 2-3 Closest to window 2 

3-A Closest to time period A 


5 


Closest to window 4 


Ranking is performed automatically within the acquisition selection 
program. 

• Acquisitions selected to meet the minimum data requirements for the basic 
decision logic 

- Seven (maximum) acquisitions selected are those ranked first within each 
biowindow and time period A. 

- Acquisitions are checked for cumulative obscured ground area. 

- Process! bility of a segment based on acquisitions depends on the bio- 
windows acquisition being in the geographic location of the segment. 

• Acquisitions selected to meet minimum data requirements for the augmented 
decision logic 

The minimum data required for spring small grains processing using the 
augmented branch are at least one acquisition from each of the following 
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three groups: (1) group 1 Includes period 0, window I, window 4, and 

period 5; (2) group 2 includes window 2, period 2-3, and window 3; and 
(3) group 3 includes period 3-A and time period A. In winter wheat areas 
such as Montana and South Dakota, an acquisition in period 0 or window 1 is 
required. 

- Six (maximum) acquisitions selected in specific order 

•• Select the maximuni number of midseason acquisitions available (up 
to 3) in the following order: window 3, window 2, period 3-A, 
period 1-2, and period 2-3. 

•• Select the maximum number of late season acquisitions available 
[up to 2 (three minus the total number of acquisitions already 
•aiected)] in the following order: window 4, time period A, 
period 5, and additional time period A. 

M Select the maximum number of early season acquisitions available [up 
to (6 minus' the total number of acquisitions already selected)] in 
the following order: window i and period 0. 

If the total number of acquisitions selected is less than 6, select 
the maximum number of additional acquisitions available [up to 
(6 minus the total number of acquisitions already selected)] in the 
following order (if not previously selected): period 1-2, period 5, 

and additional time period A. 

- Acquisitions checked for combined obscured ground area. (Obscurity must 

be less than 40 percent.) 

A segment is processible if it is processible by either augmented or basic 
decision logic. If processible by both, the basic decision logic is used. 
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A. 2. 6 BIOWINDOW MIDPOINT MODEL 




• Uses daily miniinum and maximum temperatures to calculate growing degree 
days (ODD) 

• Biowindow midpoints are defined by particular GDD values (figure A-21). 
The Julian date at which the biowindow GDD value is attained is then used 
for defining opening and closing, dates of the biowindow. 


• Biowindow definition for the basic decision logic 

Biowindow midpoints output from the crop calendar model are used to 
determine the opening and closing dates for each of the four biowindows as 
follows : 


Bio- 

window Opening date 

1 Spring wheat, 

50" planted 
minus .5 days 

2 Spring wheat, 

50% headed 
minus 10 days 

3 Spring barley, 

50% turning-to- 
ri pe minus 6 days 

4 Spring wheat, 

50% harvested 
plus 15 days 


Duration , 
Closing date days 

Spring wheat, 23 

50% planted 
plus 18 days 

Spring wheat, 20 

50% headed 
plus 10 days 

Spring barley, 12 

50% turning-to- 
ri pe plus 6 days 

Spring wheat, 15 

50% harvested 
plus 30 days 


The basic branch's decision logic also utilizes acquisitions from time 
period A. The opening date of time period A is calculated as follows: 


Time period A _ /biowindow 4 


opening date 


) /biowindow 3 \ 
Iclosing date! 

/Biowindow 3 \ 
Iclosing date/ 

endinrSate ^ “ (opening date of biowindow 4) 


^opening date 
(0.4) 
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*GDO = Growing degree days with base 32°F. 

Figure A-21.- The relationship between window/periods used in procedures SSG3B and SSS3C 
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t Biowindow definitions for the augmented decision logic 

The augmented branch's decision logic uses acquisitions within the 
remaining interim periods, a 30-day period preceding biowindow 1 (period 0) 
. and a 15-day period following biowindow 4 (period 5). Period 1-2 spans the 
time from the closing date of biowindow 1 to the opening date of 
biowindow 2. Period 2-3 is the time between biowindows 2 and 3; period 3-A 
is the time between biowindow 3 and time period A. The biowindows and 
periods have been grouped into early-season, midseason, and late-season 
acquisitions. 

• The FY 1980 U.S. Wheat/Barley Exploratory Experiment data include 49 spring 
wheat ground-truth segments from crop year 1979. 

- Results of the experiment follow: the best planting date and crop 

growth stage models correctly assigned acquisitions to biowindows 1 and 
2 seventy-three (73) percent of the time and to biowindow 3 twenty-one 
(21) percent of the time; no test was done for biowindow 4. 

• Data for the Shakedown Test of the U.S. /Canada Spring Small Grains Pilot 
Experiment include 15 ground-truth segments from crop years 1976-79. 

- Evaluation results of biowindow midpoint models (tables A-8, A-9, and 


figure A-22) 



#• Model form: 

Supporting 

Research 

Alternative 

(automated) 

Bias 

4.7 

2.8 

Root mean 

10.8 

8.4 

square error 
(RMSE) 




(Results are similar for all windows.) 

•• Alternative model form recommended, with analyst adjustment, for use 
in the baseline procedure SSG2 system in the U.S. /Canada Spring 
Small Grains Pilot Experiment. 

• The use of spring small grain window midpoints versus wheat window 
midpoints 
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TABLE A-9.- SSG ACCURACIES TO DATE 


Year 

Region 

Segment 

Accuracy, % 

Logic 

1976 

Belorussla 

7026 

75.0 

Basic 

1977 

North 

Caucasus 

7388 

90.9 

Augmented 



7336 

100.0 

Basic 



7390 

66.7 

Augmented 


Central 

7574 

100 

Augmented 


Ural s 

7911 

100 

Basic 


Orenberg 

7950 

93.6 

Augmented 




97.9 

Basic 



7959 

89.8 

Basic 




95.5 

Augmented 



7973 

90.6 

Augmented 



8021 

37.8 

Augmented 

1978 

Urals 

7902 

80.95 

Augmented 



7954 

95.5 

Augmented 



8013 




U.S.S.R 

8030 

71.70 

Augmented 



Overal 1 

92.5 

Basic 




85.3 

Augmented 


Canada 

1958 

100.0 

Basic 



3053 

74.2 

Augmented 



3175 

97.1 

Basic 



Overal 1 

98.6 

Basic 




74.2 

Augmented 








Green number 
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BIOWINOOW MIDPOINT MODEL 




• In years of late SS6, summer crops are committed in window (W) 2, and 
period A is needed to screen them out. 

• In a year of normal spring grains, summer crops are not generally 
committed; but if so, the summer crops will be screened out by W4. 


Figure A-22.- Window estimates of spring small grains. 
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Comparison of the selected FSR segments for window 2 midpoint shov^ed no 
significant difference between the wheat midpoint and the midpoint for a 
grain mixture. 


- Comparison of the two methods using a U.S. segment with more than 

30 percent barley (almost all early maturing) showed no difference in 
windows 1 and 2 and showed a 1-day difference in windows 3 and 4. 

- Very little early barley separation has been observed in the U.S.S.R. 
segments, which would tend to cause window midpoints to be too early. 

• Problems indicated from U.S.S.R. segments in experimental processing 

- Major impact on labeling: 

•• Commission of noncropland 

•• Winter wheat commission (lack of proper acquisitions) 

•• Omission of spring small grains because of insufficient green-up 
caused by poor stands, poor soil, etc. 

•• Possible window-finder failure in short spring small grains growing 
season of North Caucasus Region or because of year-to-year 
variabi 1 ity. 

- Minor impact on labeling: 

#• Summer crop commission 

•• Rapeseed commission in Canadian segments 

• Winter grain commission 

- Winter grain is in 12 of 14 U.S.S.R. segments; content 1s as high as 
108 dots; commission is as high as 37 dots; logic for software is under 
study. 

• Spring crop commission 

- Five U.S.S.R. segments with nongrain spring crops; maximum dots • 
committed to spring grains using current logic are 8 (probably 
potatoes). 
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- Two Canadian segments with rapeseed: segment 3175 has three dots, two 

of which are committed to spring grain. Crop Year 1958 has 38 dots, 35 
of which are committed to spring grain. (See no potential software 
solution. ) 

t Potential 1977 drought effects 

- Could result in serious (50-67%) omission of spring small grains in 10 
to 25 segments in the North East Urals 

- Some effect (10 percent or more omission) of spring small grains on as 
many as 33 percent of the Ural segments. 

- Possible obscuration of any barley separation that might otherwise occur 
in 50 percent or more of the Ural segments. 

• Data Assessment 

- Bellorussia, 50 percent as much potatoes as spring small grains 

- Central Region, 22 percent as much potatoes as spring small grains 
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A. 2. 7 SPATIAL/COLOR SEQUENCE (SSG4) PROCEDURE 

Figure A-23 is the spring small grains spatial/color sequence procedure (SSG4). 

• The developmental evaluation of procedure SSG4 data includes 13 segments 
from crop years 1978 and 1979, which the integrated procedure and procedure 
SSG4 have in common. 

• Results of the proportion estimation evaluations (table A-10) 

- There is little difference between the MSE of the SSG4 procedure and the 
integrated procedure. 

- The bias of SSG4 is substantially smaller than the bias of the 
integrated procedure. 

TABLE A-10.- PERFORMANCE COMPARISON OF INTEGRATED AND SSG4 PROCEDURES 





Standard 

Technology 

MSE 

Bias 

deviation 

Integrated 

40.32 

-5.0 

4.07 

SSG4 

49.56 

-2.0 

7.03 
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A. 2. 8 ERROR MODELS 

• Error models are quantified predictors of proportion estimation 
performance. 

• The models* primary purpose is to evaluate the performance in foreign areas 
with limited ground observations. 

• The models substitute for ground truth or maximal analysis. 

• Error models are useful in analysis and improvement of proportion 
estimation technology. 

• Current approach to error modeling (fig. A-24) 

- Gather performance data on all blind site segments except FSR 

- Develop data on all quantifiable sources of error 

- Construct model using half of data available 

- Validate model using other half 

- Test model in FSR 

- Apply model in IR 

• Categories of error model variables 

- Field characteristics 

- Crop-mix estimates 

- Acquisition histories 

- Meteorolog'iCiii effects 

- Crop-growth stage variability 

• Error models - conclusions from preliminary modeling activity (table A-11) 

- Prediction at segment- level is difficult. 

- Prediction at higher-level appears feasible. 
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TABLE A-11.- PRELIMINARY ERROR MODEL RESULTS FOR 197.8 
SEGMENT LEVEL (N = 31) 

Dependent Variables Independent Variables Mean R^ 

Integrated omission rate Integrated proportion estimate 

Integrated commission rate Automated proportion estimate 0.48 

Integrated proportion bias Mean size SSG fields 

Automated proportion bias Planting period duration 

Comments ; 

• Many combinations illogical 

• Little pattern among models 

APU (STRATUM) LEVEL (N = 6) 

Dependent Variables Independent Variables Mean R^ 

Integrated omission rate Lateness of last acquisition 

Integrated commission rate Moisture jointing - heading 

Integrated proportion bias Number of fields in segment 0.90 

Automated proportion biass Percent fallow + pasture signature 

Integrated RMSE Mean size SSG fields 

Automated RMSE 

Comments : 

• Most combinations logical 

• Integrated and automated bias models similar 
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Accuracy assessment. FSR approach. 
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A.2.9 HISTORICAL PROPORTION ESTIMATION 

Table A-12 presents the historical proportion estimates. 
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TABLE A-12.- HISTORICAL PROPORTION ESTIMATES (INTEGRATED LABELING) 


Crop 

A 

X 

X 

D 

% 

N 

SIGNIFICANCE 

Winter small grains; 
LACIE, Phase III 

22.9 

27,7 

-4.8 

0.8 

90 

★ 

Winter wheat: 
LACIE, Phase III 

22,0 

25.1 

-3,1 

0.7 

91 

★ 

TY, Early season (Feb.) 

19.2 

23.7 

-4.5 

1.1 

51 

★ 

TV, After harvest 

14.2 

15.1 

-0.9 

0.6 

56 


Spring small grains: 
LACIE, Phase III 

28,8 

34.9 

-6.1 

0.8 

45 

★ 

TY 

24.. 7 

28.7 

-4.0 

1.2 

38 

★ 

TY, less barley 

23.6 

28.6 

-5.0 

1.1 

38 


TY, barley 

3.5 

4.6 

-1.1 

0,6 

38 

★ 

1980 Exploratory Experiment 
(Proportional Bayesian, 
integrated labels) 

(-) 

(-) 

-3.5 

1.0 

35 

★ 

(Proportional Bayesian, 
reformatted labels) 



6.0 

5.5 

9 


Development test (SSG) 




1 



Spring wheat: 
LACIE, Phase III 

16.1 

19.3 

1 

-3.2 

1 

i 

0.8 

( 

53 

★ 

TY 

15.5 

19.9 

1 

-4.4 

1.0 

38 

★ 


Symbol definition: 

D = X - X 

N = Sample size 

Sq = Standard deviation of 0 

X = Average harvested proportion estimate 
X = Average ground-truth proportion estimate 

* = Indicates that the estimate was significantly different from the 

ground-truth proportion (a = 0.10). 
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A.2.10 HISTORICAL LABELING ACCURACY 

Table A-13 presents the historical labeling accuracy. 

« Labeling summary 

- Accurate spring small grains labeling can be obtained for individual 
U.S.S.R. segments with slight adaptations to the baseline procedure. 
Similar results are attainable for segments in FSR's. 

- Barley separation in the U.S.S.R. is not comparable to that observed in 
the U.S. 

•• There is large variability in the confidence associated with barley 
separation. 
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TABLE A-13.- HISTORICAL LABELING ACCURACY 


Crop 

Experiment | 

Accuracy, % 

Integrated Procedure 

Spring 

grains 

small 

Lacie, Phase III 

62 



Transition Year 

73 



1980 Exploratory Experiment 

80 

Barley 


Transition year 

^50 



1980 Exploratory Experiment 

■ 57 

ERIM 

Barley 


Phase II 

72 



Phase III 

86 

Reformatted Procedure 

Spring 

grains 

smal 1 

1980 Exploratory Experiment 

77 

Barley 


1980 Exploratory Experiment 

51 



Spatial/Coior SS6 Procedure 

- 

Spring 

grains 

smal 1 

Development Test 

^66 


’Barley was separated from correctly labeled spring 
small grains at a 63-percent success rate; includes 
winter wheat. 


^This is low by design since the procedure is a 
proportion estimation procedure rather than a 
labeling procedure; includes winter wheat. 
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A. 2. 11 AGGREGATION TECHNIQUES (figure A-25) 

• Evaluation of Grouped Optimal Aggregation Technique (GOAT) 

- GOAT represents at least two distinct improvements over LACIE 
technology; see figure A-26. 

M GOAT considers acreage estimate for each stratum to be a linear 

combination of direct and ratio estimates. (In both cases, ratioing 
is done based on historical data, but GOAT permits use of multiple 
years of historical data; in LACIE, only one historical year was 
used. 

M When ratioing was performed in LACIE, ratios were based on arbitrary 
collections of strata. GOAT finds optimal grouping for determining 
ratios. 

- Simulation studies indicate the following. 

•• GOAT has no procedural bias. 

•• GOAT is robust against missing data. 

•• GOAT provides good estimates of variance. 

• Improvements over LACIE technology 

- In LACIE, stratum-level acreage proportion estimates were exclusively on 
current year segment proportion estimates, even for strata containing 
only one segment. 

- Multiyear (MY) mode considers acreage proportion estimates for each 
stratum to be a function of both current and previous year direct 
estimates (fig. A-27). 

• Preliminary results 

- Tests with use of three years of North Dakota wheat data indicate that 
using MY reduces the variance of the resulting stratum- level estimates. 
Further work continues at Texas ASM University (TAMU). 
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"xa B 

™ 0 » quauty 


SINGLE Ve\R. 


'I' 

W 


GOAT 


-> WGCAT 


Figure A-25.- Progression of aggregation techniques. 
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°^I0INAL PAf.~ „ 



Figure A-26.- GOAT versus single year aggregation. 
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Figure A-27.- The multiyear (MY) model versus single year^ 
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A. 2. 12 HISTORICAL U.S.S.R. ESTIMATION 

Table A-14 presents the LACIE, Phase III, U.S.S.R. estimation accuracy for 
total wheat production. Table A-15 presents the production, area, and yields 
estimates for U.S.S.R. winter wheat for Transition Year (1978) early season. 


TABLE A-14.- LACIE, PHASE III (1977), U.S.S.R. ESTIMATION 
ACCURACY OF TOTAL WHEAT PRODUCTION 


Month 

RD, % 

CV, % 

Tolerance 
limits, % 

Significance 
level, % 

August 

-3.3 

4.3 

(-4.5, 4.0) 

50 

September 

0.3 

3.9 

(-5.6, 4.6) 

50 

October 

-0.4 

3.8 

(-5.6, 4.6) 

50 

Final 

-0.7 

3.8 

(-5.6, 4.6) 

50 


Symbol definition: 


CV * coefficient of variance 
RO = relative difference 


TABLE A-15.- TRANSITION YEAR (1978) EARLY-SEASON PRODUCTION, AREA, 
AND YIELD ESTIMATES FOR U.S.S.R. WINTER WHEAT 


Item 

LACIE. Phase III 
(March 30, 1977) 

TY 

(March 15, 1978) 

Estimate 

CV, % 

U.S.S.R. 

estimate 

RO, % 

Estimate 

CV, % 

U.S.S.R. 

estimate 

RD, % 

Area , 

(ha X 10") 

21.3 

6.3 

20.7 

-2.9 

20.7 

4.3 

23.1 

-11.0® 

Yield 

(ql/ha) 

24.3 

4.4 

25.0 

-2.9 

22.8 

2.9 

(a) 

(a) 

Production 
(MT X 10®) 

51.6 

7.0 

51.9 

- .6 

47.1 

4.0 

(a) 

(a) 


^Oate not available. 
"Difference Is Insignificant. 


Symbol definition: 

CV » coefficient of variance 

MT » metric ton 

RD » relative difference 
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• The multiyear (MY) model 

- Basic model: Y(PTS) = oj + Bg + 

Where 

Y(Pjs) = A variate transformation of the segments estimated at-harvest crop 
acreage proportion for a given year, Pts* 

ay = The stratum's transformed at-harvest crop acreage proportion for a 
given year. 

Bg = The departure from the stratum's transformed at-harvest crop 
acreage proportion for the sampled segment. 

Eyg = The total of: sampling errors, classification errors, model lack- 

of-fit, etc. 

T = The year in which the estimate, Py^, was made. 

S = The segment for which the estimate, Py^, was made. 

The estimate of the stratum's at-harvest crop acreage proportion is: 

V'' (»t) 
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A.2.13 OTHER TECHNOLOGIES 

A. 2. 13.1 Additional Technologies for Potential Inclusion 
(No plans exists for inclusion at present.) 

e Badhwar 

• Chittinneni 

• Environmental Research Institute of Michigan (ERIM) 

• Distribution labeling 

• Partial response 

A. 2. 13.2 Other Technologies - ERIM Del iveries 

• SSG Procedure 

- Not adapted to a U.S.S.R. environment 

- BLOB parameters set to optimize corn/soybean labeling; this results in a 
greater number of mixed BLOBS. (BLOB is an algorithm.) 

- Documentation not available until Febrvjary 15, 1981 

• Barley Procedure 

- Uses information about BLOB means 

- 1$ dependent upon preprocessing 

- Has no consensus at ERIM regarding non-ERIM spring small grains 
procedure interfaced with ERIM barley. 

- Unavailable documentation until February 15, 1981 

• General 

- No software for proportion estimation 


OF dJI!? *■ « 

POOR QUALITY 
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A.2.13.3 Early Season Estimation Procedure 

• Reasons for testing and evaluating of an early-season procedure during the 
U.S.S.R. Barley Exploratory Experiment follows: 

- No early-season procedure Is currently available 

r Development of early-season procedures is underway In supporting 
research, but delivery dates dt? not support the U.S.S.R. Exploratory 
Experiment (I.e., Subjective Procedure - Delivery date July , 1981; 
Objective Procedure - Delivery date May 1, 1982). 

• Alternative Solutions 

- Do a Shakedown Test prior to the pilot on the subjective procedure. 

- ‘The objective procedure could undergo preliminary testing during the 
pilot. 

- Postpone early-season procedure use until It can be tested in an 
exploratory experiment (I.e., Incorporate subjective and objective 
early-season procedures in an 1983 exploratory experiment). 

• Impact 

- New analysts will need to be trained because those in the exploratory 
experiment will have already seen the at-harvest data. 

- Involves replanning and scoping of a future exploratory experiment. 

- A demonstration of early-season procedures in the U.S.S.R. would be 
jeopardized. 

• Recommendations 

- Postpone early-season testing until the U.S.S.R. pilot experiment. 

•• A demonstration of the subjective system would be possible pending a 
recommendation from the Shakedown Test. 

e* An exploratory-type test could be run on the objective procedure If 
It Is delivered as an automated procedure. 


UHiGINAl 
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A. 3 DATA ASSESSMENT OF POOR QUALITY 

A. 3.1 LAND SAT DATA 

t Approximately 3B5 segment-years are available for processing in U.S.S.R. 
IR's. 

• Approximately 183 segment-years are available for processing in U.S. FSR's. 

• Approximately one-fourth to oie-half of the available segments are 
potentially processible for an at-harvest SSG estimate. 

• FY 197fi-79 U.S. Great Plains (USGP) Allocation Scheme 

- FY 1976 Large Area Crop Inventory Experiment (LACIE), Phase II, Neyman 
allocation to counties based on variability of total wheat acreage. 

- FY 1977 LACIE, Phase III, Neyman allocation to counties based on 
variability of total wheat production. 

- FY 1978 LACIE TY Neyman a1 vocation to refined strata based on 
variability of total wheat production. 

- FY 1979 maintained ground truth segments from 1978 in North Dakota, 
Minnesota, South Dakota (6 segments), and Montana (2 segments). 

A.3.2 U.S.S.R. HISTORICAL DATA 

• Problems with U.S.S.R. historical data 

- Some oblast definitions change from year to year. 

- Missing oblast data 

- Bad historical data 

- Need to check problems in data for the past 4 years. 

• Limited multiyear data 

The historical data are obtained from the following sources. 

0 CIA data 

- Generally reported at the oblast level 

- Contains area, yield, and production information 
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- Some data are obviously estimated and discrepancies 

- fto data are avai1abl{‘ which coincide with available 
the U.S.S.R, 

• New computerized data set 

- Oblast-level reporting 

- Harvested area information 

- 1958-1978 crop years 

- All major crops 

- Currently being evaluated for completeness and reliability 

• Official U.S.S.R. agricultural statistics 

- Ecomonic region reporting 

- Planted area 

f Reference documents 

- Oocumentation of agricultural statistics for various special areas and 
for reporting levels (i.e., national, republic, economic, regions) 

- Oocumentation on nonstatistical agricultural information such as 
winterkill, management practices, diseases, insects, and other, 

- Some documentation covering geophysical data such as soils, topography, 
and land use. 

Tables A-16 through A-21 provide the following data: blind sites for U.S.S.R. 

FSR's, intensive test sites and blind sites within the U.S. /Canada FSR's, 
segments available in U.S.S.R. IR's, barley ground-truth sites in the FSR's, 
and the FSR data set. 
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TABLE A-20.- BARLEY GROUND-TRUTH SITES IN THE FSR' 



at least five acquisitions. 

Range of barley percentages. 

Number of segments with barley and at least five acquisitions. 
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LEVEL B DATA REQUIREMENTS 


B.l GENERAL 

Data Set identification forms for the preliminary Level B requi reinents for the 
U.S.S.R. Barley Exploratory Experiment are presented along with correspondence 
dated December 18, 1980. 


ORlGlNAt PAGE IS 
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CNCINEEhtMG AND MANAGEMENT SERVICES COMPANY, INC. 


December 18, 1980 
Ref: 644-1912 


Mr, R. M. Bfzzell 

National Aeronautics and Space Administration 
Lyndon B. Johnson Space Center 
Houston. Texas 770SS 

Dear Mr. Bizzell : 

Subject: Preliminary Level B Requirements for the USSR Barley 

Exploratory Experiment 

The attached forms are to be submitted to v.:e Data Systems Branch as the 
designated method of stating requirements for the USSR Barley Exploratory 
Experiment. An explanation of some of the listed requirements and additional 
requirements not currently covered In the forms Is also attached. Due dates 
are as requested on the forms. 

Prioritization of specific deliveries and other actions should be coordinated 
with Christine Dailey, Lockheed, USSR Barley Exploratory Experiment Integrator, 
room 243, building 17, extension 4761. 

Although these are the Level B requirements, minor modifications are anticipated 
and will be submitted innedlately following the presentation of the "Stravanan" 
experiment design. 


APPROVED: 


G. Baron 
Experiment Integration Manager 

ec: JSC/L. F. Childs J. L. Dragg 

H. Prior ” 

L. Wade 
6. HcKain 

LOCKHEEO/J. J. Vaccaro w/o attch 
B. L. Carrol I'Brcc^ 

0. Marquis 

M. Pore 

L. nores// 

File 


Sincerely, , 

C. L. Dailey 
CONQURRENCEj 

F. M. Solomon, Scientific Supervisor 
Data Requirements and Support Section 


ItJONASA ROAD 1. Howtioei. 7705G • 17131 333<&4n 


A SubsidiMfY of Lockheed CbtpOfMVon 
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Std. Data Set # R2 
Task h'omber(s) 13-Ua 
Date Required APril 1 . i981 
Requester/Org. Dai lev/Lbek heed 
Date of Request 12/15/80 


REFERENCE MAP/MOSAIC REQUIR£H|NTS FORM 
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B.2 CLARIFICATION OF INFORMATION ON LEVEL B REQUIREMENTS FORMS AND 
ADDITIONAL REQUIREMENTS ~ 

B.2.1 ANALYST PACKET PREPARATION 

• All designated Level B segments need to be inventoried. 

• The following products should be available in the packets when they are 
delivered. 

1. Image Products 1 and 3 for each image on the data base should be 
mounted and screened as described in U.S,„ Wheat/Barley Pilot Experiment 
requirements letter (Ref: 644-1875). 

2. Crop calendar and weather information including historical crop 
calendar, adjusted crop calendars for wheat and barley, and the 
climagraph products for that segment. 

3. Topgraphic maps 

At least one map should be in the packet at the largest scale available 
(e.g., for U.S. segment 1:24,000 scale; for USSR 1:250,000 scale). 

4. Ancillary data 

• LAC IE ancillary data which contain cropping practices and crop 
statistics. 

• Universal Strata Descriptor. 

5. Spactra! aids which are necessary for use during initial segment 
analysis (TBD). 

• If the production film convertor (PFC) is working, new image products 
should be generated and substituted for current imagery. 

The priority for generation of image products would be: 

1. Missing images Product 1 

2. All Product 3 

3. All remaining Product I's. 
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Note ; Any segments used during the U.S. Spring Grains Pilot Experiment 

should have met this requirement. (All FSR segments are a subset of 
segments requested for the U.S. Spring Grains Pilot Experiment.) 

f All other materials currently in the packets should be removed and 
stored as described in Section 2.2, LACIE Transition Project Detailed 
Analysis Procedures, LACIE-00724, JSC-13756, May 1979. These materials 
should be stored in an area accessible to the analysts. 

B.2.2 MATERIALS TO BE AVAILABLE IN THE ANALYST WORK AREA 

• LACIE Weekly Meteorological Statistics for the U.S. and the U.S.S.R. for 
1976, 1977, and 1978. 

• Full-frame imagery. 

• Maps designated on the Reference Map/Mosaic Requirements Forms. 

B.2.3 DATA BASE PREPARATION 

• Soil Tine computation should be run on all U.S.S.R. IR segments and any 
U.S. segments not prepared for the U.S. Small Grains Pilot Experiment. 

• A merged and transformed data base should be set up for all the segments in 

this test as described in correspondence (Ref: 644-1910, October 3, 1980). 

• Image screening information obtained from the “Image Screening Form" of all 
segments used in the exploratory are to be placed on a file accessible to 
the Acquisition Selection Program. 

• Crop calend'^r information should be on a file, accessible and compatible 
with the Acquisition Selection Program. 

• A directory file as described in correspondence (Ref: 644-1910) should be 

available for the segments specified on the Level B forms. 

• Generate data from CLASSY clustering prior to the beginning of segment 
analysis; due date - April 1, 1981. 
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All of the preceding requests need to be impleniented and their existence 
verified on the JSC system by February 15, 1981 (except CLASSY ryns which are 
due April 1, 1981). 

Note ; The Laboratory for Applications of Remote Sensing (LARS) system should 
be used as a back-up until the new system becomes available. 

B.2.4 DATA HANDLING 

• The procedure which is currently outlined on the Sensor Data Form No. 2 
includes the current procedures to be used in the U.S. Pilot and 
Exploratory Experiment and TBD candidate technology. 

• All candidate technology which has been requested but remain TBD will be 
identified as soon as possible, and the final group of technology to be 
worked will be identified following the presentation of a preliminary 
experiment design. 
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1981 U.S.S.R. FSR SEGMENT ALLOCATION 


APPENDIX C 


1981 U.S.S.R. FSR SEGMENT ALLOCATION 

Data sheets for the 1981 U.S.S.R. FSR segment allocations are Included In this 
appendix. 
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